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Abstract: The accuracy of a novel community health worker anti- 
retroviral therapy eligibility assessment tool was examined in 
corrmiunity members in Blantyre, Malawi. Nurses independently 
performed World Health Organization (WHO) staging and CD4 
counts. One hundred ten (55.6%) of 198 HIV-positive participants 
had a CD4 count of <350 cells per cubic millimeter. The community 
health worker tool significantly outperformed WHO clinical staging in 
identifying CD4 count of <350 cells per cubic millimeter in terms of 
sensitivity (41% vs. 19%), positive predictive value (75% vs. 68%), 
negative predictive values (53% vs. 47%), and area under the receiver- 
operator curve (0.62 vs. 0.54; P = 0.017). Reliance on WHO staging is 
likely to result in missed and delayed antiretroviral therapy initiation. 
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INTRODUCTION 

The 2010 World Health Organization (WHO) guide- 
lines recommend that HIV-infected adults with a CD4 count 
of <350 cells per cubic millimeter be initiated onto antiretro- 
viral therapy (ART).' Availability of CD4 count measurement 
remains low in much of sub-Saharan Africa. For example, in 
Malawi, only 12% of HIV care facilities have functional 
on-site capacity for CD4 count measurement.^ Where facili- 
ties for CD4 count are not available, individuals assessed, as 
being in WHO clinical stage 3 or 4, should receive ART.' 

The WHO clinical staging system for HIV was 
introduced in 1990^ and was based on previous highly spe- 
cific WHO case definitions for AIDS that were used in the era 
before the availability of reliable and rapid HIV diagnostic 
tests."'^ Although the WHO clinical staging system has good 
prognostic value in predicting death* and response to ART,^ it 
was not originally designed to be used for as screening tool 
for ART eligibility. Previous studies*"" have shown the sen- 
sitivity of nurse-performed WHO stage 3 or 4 assessment in 
identifying CD4 count of <350 cells per cubic millimeter to 
be low (between 18% and 65%), potentially resulting in 
missed and delayed ART initiation.'^ 

We previously found that eligibility assessments 
were a frequent cause of dropout before ART initiation,'^ 
with health workers finding WHO staging overly complex 
and time consuming, and individuals reporting that com- 
pletion of CD4 count measurement required multiple fa- 
cility visits."'"'^ With task shifting of HIV testing to 
community health workers (CHWs)"" and moves toward 
"test-and-treat,"'' eligibility assessments should be com- 
pleted immediately following HIV diagnosis to maximize 
opportunities for linkage to ART. We therefore developed 
a brief CHW tool for assessing ART eligibility and com- 
pared its accuracy with that of with the current WHO clin- 
ical staging system. 



METHODS 

Study Design 

An accuracy study nested within a cluster-randomized 
trial that was investigating cotnmunity-based approaches to 
HIV diagnosis and treatment.'* 
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Accuracy of CHW Tool for ART Eligibility 



ART Eligibility Guidelines and Development 
of Novel ART Eligibility Screening Tool 

National ART eligibility guidelines in Malawi matched 
WHO recommendations (ie, CD4 count of <350 cells/mm^ 
or in WHO stage 3 or 4)^ 

The CHW tool was designed based on the WHO "Ask, 
Look, Classify, Act" approach incorporated within guidelines 
for the Integrated Management of Adult and Adolescent Ill- 
ness.^" Questions were constructed for inclusion in the CHW 
tool from existing components of the WHO clinical staging 
system^'; with additional vaUdated questions that have pre- 
viously assessed to be important predictors of ART eligibility 
(self-rated general health "'■^^) and advanced immunosuppres- 
sion (self-rated skin or hair change). All questions 
included had to be easily understood and performed by 
CHWs and not require further laboratory investigations. The 
CHW tool classifies HIV-positive patients into 2 groups: 
those who are ART eligible (analogous to WHO stage 3 or 
4) and those who are not ART eligible (analogous to WHO 
stage 1 or 2). Individuals who responded positively to any of 
the following items were classified as ART eligible: 

• reports fever or weight loss or diarrhea for all of last 
month,^' 

• enough weight loss to affect fit of clothing,^' 

• sore mouth or swallowing,^' 

• treated for TB or any other infection (excluding malaria and 
"flu") in the last year,^' 

• self-rated change in skin or hair,^''"^' 

• unable to carry out daily duties (work, go to school, do 
housework),^' and 

• self-rated general health poor or fair,'"* 

Study Site and Population 

Adult (>16 years) residents of 3 high-density urban 
townships in the north west of Blantyre, Malawi, were pro- 
vided with access to home HIV self-testing through CHWs. 
CHWs were adult residents selected by a participatory 
approach and who received training in HIV testing and coun- 
seling (HTC). CHWs performed the novel ART eligibility 
assessment tool for individuals who disclosed a positive 
self-test result. HIV-positive individuals received a home visit 
from a nurse who independently performed confinnatory 
HTC, TB screening (symptom screen and sputum collection 
for smear and culture if positive), WHO clinical staging 
(without reference to the result of the CHW tool assessment), 
and drew blood for CD4 count. ART eligible participants (on 
national criteria) received offer of home or facility ART ini- 
tiation and nonehgible participants were referred to pre-ART 
clinics at local health centers. 

Laboratory Methods 

Blood samples were analyzed using the Partec Cyflow 
SL-3 platform (Partec, Gorlitz, Germany). 

Statistical Methods 

Sensitivity, specificity, positive predictive value (PPV), 
and negative predictive value (NPV) were calculated for both 
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the CHW tool and WHO clinical staging using CD4 count 
of <350 cells per cubic millimeter as the referent. As WHO 
has recently proposed expanding ART eligibility to include 
individuals with CD4 count of <500 cells per cubic millime- 
ter,^'^'' we repeated analysis with this criteria as the referent. A 
sample size of 191 patients was required to estimate sensitiv- 
ity and specificity with a precision of ±10% at the P = 0.05 
level.^^ Statistical analysis was done using Stata 12.1 
(Statacorp, College Station, TX). 

Ethical Approval 

The research ethics committees of the College of 
Medicine of Malawi, Liverpool School of Tropical Medicine, 
and London School of Hygiene and Tropical Medicine 
granted ethical approval for the parent cluster-randomized 
trial. All participants gave written informed consent to 
participate or witnessed thumbprint consent if illiterate. 

RESULTS 
Baseline Characteristics 

Between February and November 2012, 198 of 206 
HIV-positive adult community members were recruited to the 
study (96.1% participation). Men comprised 42% (84/198) of 
participants and were younger, had a higher level of 
schooling, and were more likely to have salaried employment 
than women (Table 1). 

Outcomes of ART Eligibility Assessments 

A total of 31 of 198 (15.6%) individuals were assessed 
to be in WHO stage 3 or 4 by nurses, with men significantly 
more likely to be in the more advanced WHO stage 3 or 4 
than women [19/84 (22.6%) vs. 12/114 (10.5%) respectively; 
P = 0.021]. Men also had a significantly lower median CD4 
count at assessment compared with women [262 cells/mm-' 
(mterquartile range, IQR: 148-373) vs. 374 cells/mm^ (IQR: 
195-512); P = 0.002]. A total of 110 of 198 (55.6%) partic- 
ipants had a CD4 count below the Malawian national ART 
eligibility threshold of <350 cells per cubic millimeter. 

Performance of CHW Tool in Identifying CD4 
count of <350 Cells per Cubic Millimeter 

One participant did not have the CHW tool completed 
by the CHW, leaving 197 participants who were analyzed 
with complete data for outcome of CHW tool, CD4 count, 
and WHO clinical staging assessment. 

The sensitivity of the CHW tool (41.3%; 95% CI: 
31.9% to 51.1%) was substantially higher than that of 
nurse-performed WHO clinical stage 3 or 4 assessment 
(19.1%, 95% CI: 12.2% to 27.7%) in identifying CD4 
count of <350 cells per cubic millimeter (Table 2). The 
specificity of the CHW tool was 83.0% (95% CI: 73.4% to 
90.1%) compared with 88.6% (95% CI: 80.1% to 94.4%) 
for WHO clinical stage 3 or 4 assessment. The area under 
the curve (AUC) was significantly higher for the CHW 
tool (0.62; 95% CI: 0.56 to 0.68) compared with WHO 
clinical stage 3 or 4 assessment (0.54; 95% CI: 0.49 to 
0.59; P = 0.017). 
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TABLE 1. Characteristics of Study Participants 



Men Women P* 



Total 


84 




114 






Age, yrs (mean, SD) 


37 


8.6 


31 


8.3 


<0.001 


Pregnant (if female) 


n/a 




2 


1.8% 




Breastfeeding (if female) 


n/a 




19 


16.7% 




Body mass index, kg/m^ 


20 


2.8 


22 


4.1 


<0.001 


(mean, SD) 












Marital status 












Married/cohabiting 


61 


72.6% 


73 


64.0% 


0.023 


Never married 


9 


10.7% 


5 


4.4% 




Widowed/separated/ 


14 


16.7% 


36 


31.6% 




divorced 












Ever lost a spouse to death 


16 


19.3% 


26 


22.8% 


0.569 


Able to read a one page letter 


78 


94.0% 


97 


85.8% 


0.069 


or newspaper 












Receives a regular salaried 


31 


40.3% 


8 


8.2% 


<0.001 


income 












Highest level of education 












No schooling 


3 


3.6% 


1 


0.9% 


0.042 


Primary 


39 


46.4% 


75 


65.8% 




Secondary 


40 


47.6% 


36 


31.6% 




Higher 


2 


2.4% 


2 


1.8% 




Ever previously tested for HIV 


46 


56.1% 


85 


75.9% 


0.004 


Tested for HIV in preceding 12 


12 


14.5% 


24 


22.2% 


0.174 


months 












Ever previously tested HTV 


20 


23.8% 


39 


34.5% 


0.105 


positive 












Tested as a couple 


25 


29.8% 


28 


24.6% 


0.414 


Self-rated general health 












Excellent 


8 


9.5% 


10 


8.8% 


0.556 


Good 


51 


60.7% 


75 


66.4% 




Fan- 


22 


26.2% 


27 


23.9% 




Poor 


3 


3.6% 


1 


0.9% 




WHO clinical stage 












Stage 1 or 2 


65 


77.4% 


102 


89.5% 


0.021 


Stage 3 or 4 


19 


22.6% 


12 


10.5% 




CD4 count, cells/mm^ 


262 


148-373 


374 


195-512 


0.002 


(median, IQR) 












CD4 count group, cells/mm^ 












>500 


11 


13.1% 


30 


26.3% 


0.015 


a350 to <500 


15 


17.9% 


32 


28.1% 




>200 to <350 


27 


32.1% 


23 


20.2% 




>50 to <200 


30 


35.7% 


26 


22.8% 




<50 


1 


1.2% 


3 


2.6% 




CHW ART eligibility assessment tool 










ART eligible 


1128 


33.7% 


32 


28.1% 


0.394 


Not ART eligible 


55 


66.3% 


82 


71.9% 





* The x' test for categorical data, t test for comparison of means, and the rank sum 
test for comparison of medians. 



Unexpectedly, there were differences in test perfor- 
mance when resuhs were stratified by gender. The sensitiv- 
ity and specificity of the CHW tool were higher among men 
(45.6% and 92.3%, respectively) than among women 
(36.5% and 79.0%, respectively). For both men and women, 
however, the PPV (92.9% vs. 78.9% and 59.4% vs. 50.0%, 
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respectively) and NPV (43.6% vs. 33.8% and 59.8% vs. 
54.9%, respectively) of the CHW tool was higher than that 
of WHO clinical staging. The largest AUC was observed for 
men identified as ART eligible by the CHW tool (AUC = 
0.69), which was significantly higher than for WHO stage 3 
or 4 assessed men (AUC = 0.55; P = 0.016). For women, 
although the AUC for the CHW tool (AUC = 0.58) was 
greater than for WHO stage 3 or 4 assessment (AUC = 
0.51), the difference was relatively small and did not reach 
statistical significance {P = 0.104). 

Using CD4 count of <500 cells per cubic millimeter as 
the referent, the CHW tool similarly outperformed WHO 
clinical staging assessment in terms of sensitivity, PPV, 
NPV, and AUC (Table 2), although performance of both tools 
was worse than at the CD4 count of <350 cell per cubic 
millimeter cutoff. 



DISCUSSION 

The main finding from this study was that our brief 
CHW tool significantly outperformed the current WHO 
clinical staging system for identifying ART eligible indi- 
viduals with CD4 count of <350 cells per cubic millimeter. 
The ease with which we were able to improve on the current 
WHO staging system emphasizes the need for improved 
tools that can be used by commvinity and facility-based 
health workers to ensure "same-day, same-site" eligibility 
assessments and prompt linkage to ART. Moreover, the low 
sensitivity and high risk of misclassification when using the 
standard WHO clinical staging system in this and other 
studies^"" questions its appropriateness as an ART eligibil- 
ity assessment tool, especially with increasing CD4 count 
thresholds.^* 

The CHW tool was found to be significantly more 
accurate than WHO clinical staging assessment in terms of 
sensitivity, PPV, and NPV. In contrast to the WHO clinical 
staging system, which, strictly followed, requires nurses or 
doctors to undertake a complex and time-consuming assess- 
ment,'" our CHW tool was easily performed in the commu- 
nity and could be used in settings without sophisticated 
laboratories. The poor performance of nurse WHO clinical 
staging system in this and other studies**"" is therefore not 
a factor of the capability of nurses. Instead it is a reflection of 
the inherent limitations of the WHO screening tool for iden- 
tifying ART eligibility, a task for which it was never origi- 
nally intended. 

The CD4 count is the gold standard for determining 
ART eligibility, but access to laboratory measurement is 
problematic in most resource-limited settings.^' Point of care 
CD4 count measurement has the potential to negate some of 
these issues and improve retention in pre-ART care,^" but it 
is not yet widely available. Where CD4 count is not avail- 
able at the point of HIV diagnosis, improved eligibility 
assessment tools such as ours could allow an immediate 
decision on ART initiation to be made, facilitating "test- 
and-treat" delivery of ART. 

The accuracy of the CHW tool was noted to be higher 
among men than women. This may reflect the relatively more 
advanced immunosuppression of men compared vnth women 
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Accuracy of CHW Tool for ART Eligibility 



TABLE 2. Accuracy of CHW Tool and WHO Clinical Stage 3 or 4 for Predicting CD4 count of <350 and <500 Cells per Cubic 

Millimeter 

Versus CD4 <. 350 ceUs/mm^ Sensitivity (%, 95% CI) Specificity (%, 95% CI) PPV (%, 95% CI) NPV (%, 95% CI) AUC (%, 95% CI) 

Overall 

WHO clinical stage 3 or 4 
CHW tool 
Among men 

WHO clinical stage 3 or 4 
CHW tool 
Among women 

WHO clinical stage 3 or 4 
CHW tool 



19.1 (12.2-27.7) 
41.3 (31.9-51.1) 

25.9 (15.3-39.0) 
45.6 (32.4-59.3) 

11.5 (4.4-23.4) 
36.5 (23.6-51.0) 



88.6 (80.1-94.4) 
83.0 (73.4-90.1) 

84.6 (65.1-95.6) 
92.3 (74.9-99.1) 

90.3 (80.1-96.4) 
79.0 (66.8-88.3) 



67.7 (48.6-83.3) 
75.0 (62.1-85.3) 

78.9 (54.4-93.9) 
92.9 (76.5-99.1) 

50.0 (21.1-78.9) 
59.4 (40.6-76.3) 



46.7 (39.0-54.6) 
53.2 (44.6-61.9) 

33.8 (22.6-46.6) 
43.6 (30.3-57.7) 

54.9 (44.7-64.8) 
59.8 (48.3-70.4 



0.54 (0.49 0.59) 
0.62 (0.56-0.68) 

0.55 (0.46-0.64) 
0.69 (0.61-0.77) 

0.51 (0.45-0.57) 
0.58 (0.49-0.66) 



Versus CD4 <. 500 ceUs/mm^ Sensitivity (%, 95% CI) Speciflcity (%, 95% CI) PPV (%, 95% CI) NPV (%, 95% CI) AUC (%, 95% CI) 

Overall 
WHO clinical stage 3 or 4 
CHW tool 
Among men 

WHO clinical stage 3 or 4 
CHW tool 
Among women 

WHO clinical stage 3 or 4 
CHW tool 



15.9 (10.6-22.6) 
33.3 (26.0-41.3) 

23.3 (14.2-34.6) 
38.9 (27.6-51.1) 

9.5 (4.2-17.5) 
28.6 (19.2-39.5) 



85.4 (70.8-94.4) 

80.5 (65.1-91.2) 

81.8 (48.2-97.7) 
100 (71.5-100) 

86.7 (69.3-96.2) 
73.3 (54.1-87.7) 



80.6 (62.5-92.5) 

86.7 (75.4-94.1) 

89.5 (66.9-98.7) 
100 (87.7-100) 

66.7 (34.9-90.1) 
75.0 (56.6-88.5) 



21.0 (15.1-27.9) 

24.1 (17.2-32.1) 

13.8 (6.5-24.7) 
20.0 (10.4-33.0) 

25.5 (17.4-35.1) 
26.8 (17.6-37.8) 



0.51 (0.44-0.57) 
0.57 (0.50-0.64) 

0.53 (0.40-0.65) 
0.69 (0.64-0.75) 

0.48 (0.41-0.55) 
0.51 (0.42-0.60) 



CI, confidence interval. 



in the study, consistent with findings from HIV care 
programmes in sub-Saharan Africa showing that men initiate 
ART later than women.^^ Women may have also been less 
likely to report symptoms included in the tool. Future, larger 
studies should examine the performance of CHWs tool 
among men and women across CD4 count strata. 

Our CHW tool was designed pragmatically to combine 
ease of use by CHWs with questions that were hoped would 
have a high discriminatory power. Component parts of the 
tool were not captured and so we were unable to assess the 
contribution of individual items to the tool performance. As 
such, it is possible that the number of items could be reduced, 
or the sensitivity and specificity improved beyond that 
reported here. Future research should attempt to refine the 
tool by examining the contribution of the included items to 
the test performance and evaluating any modifications. 

In conclusion, this study has shovm that a novel 
screening tool performed by CHWs can distinguish ART 
eligible from ART ineligible HIV-positive individuals. This 
could be added to HTC counselors' posttest evaluations, al- 
lowing same-day, same-site HIV diagnosis and ART eligibil- 
ity assessment. The accuracy of the CHW tool significantly 
outperformed WHO staging by a nurse and the ease with 
which we were able to improve on the current WHO clinical 
staging system shows the urgent need for improved ART 
eligibility assessment tools. 
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